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@ Multiple call control method in a multimedia conferencing system. 

@ A method for use in a multimedia conferenc- 
ing arrangement to control multiple concurrent 
calls where each call comprises one or more 
channels. A first call among a first set of user 
stations' and a second call among a second set 
of user stations are merged into a single call 
comprising a plurality of channels among at 
least three user stations from the first and 
second sets. The plurality of channels of the 
single, merged call corresponds to a combi- 
nation of channels of the first and second calls 
and may include a signalling channel, a voice 
channel, and one data channel for each data 
channel of the first and second calls. The 
method is also applicable to calls including 
image or video channels. 
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Technical Field 

This invention relates to communication con- 
ferencing. 

5 

Background of the Invention 

U. S. Patent 4.905,231, issued to W. F. Leung et 
al. on February 27, 1 990, discloses a multimedia com- 
munication system using a single packet switching 10 
network for the various media based on what is refer- 
red to as a multimedia virtual circuit A virtual circuit 
is a packet-switched communications path between 
two endpoints. A network call set up procedure 
establishes a virtual circuit and an associated virtual 15 
circuit identifier based on a destination address. To 
route successive packets, the network needs only the 
virtual circuit identifier. In a multimedia virtual circuit, 
a virtual circuit is divided into multiple virtual channels 
at the workstations. Each channel represents a diffe- 20 
rent information medium and has a separate channel 
identifier; the composite multimedia virtual circuit rep- 
resents the resulting multimedia call. As the various 
multimedia call sources generate traffic, the works- 
tation multiplexes the packetized traffic onto a single 25 
network virtual circuit The destination workstation 
demultiplexes, this traffic back into multiple channels, 
which it routes to a telephone, speaker, file or other 
destination. This process preserves the temporal 
ordering of the information streams so that in a voice 30 
and video call, for example, the voice corresponds to 
the mouth movement in the video picture. The mul- 
timedia virtual circuit also provides a simple way to let 
a multimedia call destination associate the multiple 
media. A single incoming call notification arrives from 35 
the network to announce a call. The workstations then 
exchange signaling information over a virtual channel 
of the multimedia virtual circuit to set up all the neces- 
sary channels and to route them to the appropriate 
devices. 40 

The Rapport multimedia conferencing system, 
disclosed in the S. R. Ahuja et al. article, The Rapport 
Multimedia Conferencing System,* Proceedings of 
Conference on Office Information Systems, March 
1988, supports interactive, real-time distributed con- 45 
ferences among two or more people Executing on 
personal workstations interconnected by separate 
data and voice networks, the Rapport system pro- 
vides basic mechanisms to create, manage, and ter- 
minate conferences. The system provides an so 
environment in which many types of meeting can take 
place, including telephone conversations, discussion 
among colleagues, and lectures. Existing workstation 
programs can be used during a conference to produce 
and edit data and displays for conferees. Rapport is 55 
designed to help people emulate face-to-face confer- 
ences as closely as possible with their workstations. 
However, the reliance on separate networks for the 



different media (data and voice) substantially compli- 
cates the control of conference calls. Although con- 
current participation in many conferences is possible, 
a user is only able to communicate on one call at a 
time. A Rapport system user may participate in many 
conferences concurrently by switching among con- 
texts at the click of a mouse button. This is equivalent 
to being able to walk in and out of several meeting 
rooms (and one's office) instantly. It is anticipated that 
this capability will encourage users to keep many con- 
ferences active for long periods of time in much the 
same fashion as the use of screen windows allows 
one to keep many programs and files active with the 
present data networks. One such long-lived confer- 
ence might be an intercom connection between a 
manager and a secretary. Others might be among the 
collaborators in a design project or the authors of a 
paper. It is anticipated that once the capability for mul- 
tiple concurrent calls is provided, it will be useful to 
merge and split such calls. For example, the manager 
may ask the secretary to join a design project confer- 
ence from time to time to assist the project team. The 
effective control of a plurality of a multimedia calls 
poses a technical problem, particularly in the Rapport 
system arrangement comprising several separate 
networks, but also for the system of the above-refer- 
enced U.S. Patent 4,905,231 employing a single net- 
work for all media. 

Solution 

This problem is solved and a technical advance 
is achieved in accordance with the principles of the 
invention in a method for controlling multiple concur- 
rent calls where each call comprises one or more 
channels. A first call among a first set of user stations 
and a second call among a second set of user stations 
are, for example, merged into a single call comprising 
a plurality of channels among, significantly, at least 
three user stations from the first and second sets. The 
plurality of channels of the single, merged call corres- 
ponds to a combination of channels of the first and 
second calls and, aiustrauVely, may include a signal- 
ling channel, a voice channel, and one data channel 
for each data channel of the first and second calls 
(FIG. 1). The method is also applicable to calls includ- 
ing image or video channels. 

In an aiustrative embodiment a user interface is 
provided such that the first call is established in res- 
ponse to user e.g., input via a mouse button, and, in 
addition, representations, e.g., face icons corre- 
sponding to users of the first set of user stations, are 
diplayed in a window of a display (FIG. 17). The sec- 
ond call is established in similar fashion among the 
second set of user stations. The merging of the two 
calls is effected in response to user input at a user sta- 
tion that is a member of both the first and second sets. 
As a result of the merge, representations correspond- 
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ing to the user stations of the first and second sets are 
displayed in a window for the merged call. In addition, 
representations each corresponding to one of the 
plurality of channels of the merged call are also dis- 
played in the window. Icons for a voice channel, a data s 
channel being used for a character-based application 
ksh, and a data channel being used for a graphical- 
based application EZ, are illustrated in FIG. 19 in a 
representation of a conference room. The data com- 
munication applications themselves are displayed in 10 
other windows. An access control mechanism pro- 
vides selective access for a user station to transmit 
information to and receive information from the 
plurality of channels of the merged call. The plurality 
of channels of the merged call comprise virtual chan- is 
nels of a multicase connection through a packet 
switching network. A merge request is transmitted 
from from one user station via the network to each of 
the other user stations of the first and second set. The 
merging is initiated in response to receipt of a favor- 20 
able reply from each of the other user stations. 

In a method of the invention, a single call, com- 
prising a plurality of channels, is provided among a set 
of user stations. The single call is split into a first call 
among a first subset of user stations and a second call 25 
among a second subset of user stations. The first call 
includes channels corresponding to a first subset of 
the channels of the single call and the second call 
includes channels corresponding to a second subset 
of the channels of the single call (FIG. 2). 30 

A further method of the invention is used in an 
arrangement comprising a communication network 
interconnecting a plurality of user stations. A first call, 
comprising at least a voice channel, is provided 
among a first set of user stations, and a second call, 35 
comprising at least a voice channel, is provided 
among a second set of user stations. One user station 
is a member of both of the first and second sets. The 
one user station enables a user to communicate via 
both of the first and second calls at the same time. 40 

Illustratively, the one user station comprises a 
voice bridge, implemented, for example, using a 
plurality of voice packet/analog voice converters, and 
a conventional analog voice bridge. A user is able to 
communicate on two (or more) calls at a time. The 45 
user is able to transmit voice concurrently on the voice 
channels of both of the first and second calls. The user 
is also able to transmit voice on the voice channel of 
one call and to concurrently receive voice on the voice 
channel of the other call. Further, the user is able to so 
receive voice concurrently on the voice channels of 
both of the first and second calls. 

In a further method of the invention, the network 
provides a first call among a first set of user stations 
and a second call among a second set of user sta- 55 
tions. At least one user station, which is a member of 
both of the first and second sets, includes a display. 
Representation corresponding to the first set of user 



stations art displayed in a first window for the first call, 
and concurrently, representations corresponding to 
the second set of user stations are displayed dis- 
played in a second window for the second call. A user 
may record a first call while communicating on a sec- 
ond call. 

Drawing Description 

FIG. 1 is a diagram illustrating the merger of two, 

multi-channel calls into a single call; 

FIG. 2 is a diagram illustrating the splitting of a 

single, multi-channel call into two calls; 

FIG. 3 is a diagram of a multicast packet switching 

network; 

FIG. 4 is a diagram of an individual multicast 
packet switch in the network of FIG. 3; 
FIG. 5 is a diagram illustrating a strong packet 
sequencing condition; 

FIG. 6 is a diagram illustrating the packet switch- 
ing process for a multicast connection through a 
multicast packet switch; 

FIG. 7a-FIG. 7c illustrate three data packet flow 
patterns within a multicast packet switch; 
FIG. 8a-FIG. 8d are diagrams used in describing 
properties of data packet flow patterns; 
FIG. 9a-FlG. 9c and FIG. 10a-FIG. 10b are diag- 
rams used in describing properties of vanPla mul- 
ticast connections; 

FlG.11a-FlG.11c are diagrams used in describ- 
ing properties of strong multicast connections; 
FIG. 12 and FIG. 13 are diagrams of two mul- 
timedia conferencing arrangements (the FIG. 13 
arrangement is a particular exemplary embodi- 
ment referred to herein as conference system 
1000, which implements illustrative multiple call 
control methods of the invention); 
FIG. 14 illustrates the use of the connector and 
the virtual circuit in sharing a character-based 
application program; 

FIG. 1 5 illustrates the sharing of a graphical pro- 
gram by two parties; 

FIG. 16-F1G. 24 illustrate various menus, window 
and icons for effecting conference calls and 
merging and splitting operations in accordance 
with a user interface for the conferencing arrange- 
ment of FIG. 13; and 

FIG. 25 illustrates a two-phase protocol used for 
merging and splitting multi-channel calls in the 
conferencing arrangement of FIG. 13. 

Detailed Description 

Multicast Connections 

FIG. 3 shows a multicast packet switching net- 
work which consist of multicast packet switches 
(MPSs) and network interfaces (NIs). 
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To achieve nigh-speed transmission, the multi- 
cast packet switching network is based on fast packet 
technology (described in J.J. Degan et al., "Fast 
Packet Technology for Futures Switches - , AT&T 
Technical Journal, Vol. 68, No.2, p. 36-50, 1989), hav- 
ing the following attributes: (a) Link-by-link error and 
flow control is eliminated. Thus, the required field in 
the data link header is for the logical channel number 
(LCN), which is used for routing packets through the 
multicast packet switching network. An LCN for each 
link within a connection is decided at connection setup 
time; (b) Edge-to-edge error control can be incorpo- 
rated within the multicast packet switching network on 
a per-connection basis; and (c) The multicast packet 
switching network provides internal connection-orien- 
ted services that support high-bandwidth applications 
very efficiently. In such networks, the required link 
bandwidth and the end-to-end delay for a multicast 
application are independent of the number of users. 
AJso, the network performance will not degrade as the 
number of users increases. These advantages pro- 
vide a solid foundation for the multicast packet switch- 
ing network as a vehicle for supporting various 
multicast applications, especially, interactive mul- 
timedia multi-party conferencing. 

A multicast packet switch is composed of a switch 
fabric, a switch processor, and switch interfaces (Sis), 
as shown in FIG. 4. The switch fabric is capable of 
duplicating an incoming packet and routing the copies 
to desired outgoing ports. An exemplary multicast 
packet switch is disclosed in the U.S. patent appli- 
cation of K.T. Teraslinna et al., serial number 
07/412,952, assigned to the assignee of the present 
invention. 

[Definition 2.1]: Whith multiple input streams each 
destined to multiple outputs, a switch fabric is said to 
have 

a. the weak sequencing (WS) property, if it only 
guarantees point-to-point sequential transfer 
from each input port to any of its output ports; or 

b. the strong sequencing (SS) property, if those 
output ports receiving two or more common 
inputs have identical interleaved packet streams 
with respect to the packet streams from the com- 
mon input ports. For example, in FIG. 5, the two 
subsequences of outgoing packet streams at 
switch interfaces O and E (or switch interfaces E 
and F) containing {bj and {q} (or {aj and (bj) are 
identical. 

A multicast packet switch will be represented as 
w-MPS (or s-MPS) if its switch fabric has the weak 
sequencing (or strong sequencing) property. 

In general, different links within a multicast con- 
nection may use different LCNs. Thus, each switch 
interface maintains a packet translation table (PTT) 
and a multicast control table (MCT) to store routing 
information about those multicast connections. Each 
entry of a packet translation table, indexed by an 



incoming LCN, contains a multicast connection num- 
ber (MCN) and a switch header. On the incoming link, 
the MCN field stores the MCN assigned to a multicast 
connection during connection setup, the switch 
5 header identifies a set of outgoing links involved in a 
multicast connection, which is used for packet dupli- 
cation and routing through a switch fabric. Each entry 
of the multicast control table, indexed by a MCN, con- 
tains the LCN chosen for the outgoing link within a 
10 multicast connection. 

FIG. 6 illustrates the data packet switching pro- 
cess through a multicast packet switch for a multicast 
connection. An incoming packet accesses the packet 
translation table by LCN a at switch interface A. 
15 Switch interface A then replaces LCN a in the packet 
header by the stored MCN m and prepends the stored 
switch header to the packet for packet duplication and 
routing. Each outgoing packet uses MCN m in its 
header to access the multicast control table at the out- 
20 going switch interface and obtains an outgoing LCN. 
Switch interface B and switch interface C then replace 
MCN m in the packet header by LCN b and LCN c, re- 
spectively. Finally, the switch header of each outgoing 
packet will be stripped off at the outgoing switch inter- 
ns face before it leaves. 

[Lemma 1]: Any arbitrary data packet flow pattern 
(DPFP) within a multicast packet switch can be 
achieved. 

<Proof>: Given a set of switch interfaces, with an 

30 LCN chosen for each switch interface, it is clear that 
the direction of data packet flow among these switch 
interfaces can be controlled by writing suitable infor- 
mation into their packet translation tables and multi- 
cast control tables. 

35 FIG. 7 aiustrates three natural data packet flow 

patterns within a multicast packet switch: (a) point-to- 
multipoint, (b) point-to-multipoint with upstream 
capability, and (c) multipoint-to-multipoint. They will 
be referred to as pattem-1, pattem-2 and pattem-3 

40 data packet flow patterns, respectively. 

The switch processor (FIG. 4) establishes and 
disconnects switched multicast connections across 
the multicast packet switching network. 

A network interface (FIG. 3) provides an access 

45 point to the multicast packet switching network for 
various networks and equipments, e.g., user stations, 
connected to ft. It is responsible for protocol/speed 
conversion, packet assembly/disassembly, signaling, 
etc. It also provides an edge-to-edge flow/error con- 

50 trol across the multicast packet switching network on 
a per-connection basis. 

A source-based multicast routing method is used 
to perform multicast connection setup. This method 
can be applied to both switched multicast connection 

55 setup and multicast connectionless packet routing. 

For each multicast packet switch in the multicast 
packet switching network, several spanning trees 
rooted at this multicast packet switch are generated. 
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A unique global tree number (TN) will be assigned for 
each tree. Based on these trees, multicast routing 
tables (MRTs) are established at each multicast 
packet switch during network initialization- The size of 
the multicast routing tables depends on the number of 
multicast packet switches in the multicast packet 
switching network and the number of trees. Therefore, 
a tradeoff between the number of trees and memory 
space required at each multicast packet switch is 
made. These tables may be updated dynamically. 
However, it should be done infrequently. The advan- 
tage of using multiple spanning trees is to provide 
alternate multicast routes such that the connection 
completion rate can be improved. Under normal situ- 
ations, the connection control packets for establishing 
or disconnecting a connection progress forward from 
the source multicast packet switch to the next desti- 
nation multicast packet switches. They may need to 
crankback to the source multicast packet switch for 
finding alternate spanning trees when some multicast 
packet switch belonging to the chosen spanning tree 
rejects the new connection setup for some reason. 

The basic connection setup procedure is as fol- 
lows. When a connection setup packet arrives at the 
source multicast packet switch, the switch processor 
chooses a tree number, among a set of tree numbers 
which correspond to those spanning trees rooted at 
this multicast packet switch, based on a load sharing 
method. Based on the destination set in the packet 
and the multicast routing table indexed by the chosen 
global tree number, the switch processor checks if the 
appropriate necessary and sufficient conditions des- 
cribed in detail herein are met to determine whether 
the multicast connection that would be established 
would be usable to effect communication in accord- 
ance with a specified transmission matrix and meeting 
a given packet sequencing condition. If the check is 
positive, the switch processor then partitions the des- 
tination set into severai subsets; each subset will use 
a different outgoing link. By putting the chosen tree 
number in the packet and masking off all other desti- 
nation addresses in the destination set except those 
in the corresponding subset, a modified copy of the 
original connection setup packet is then generated 
and sent to each desired outgoing link. In addition, the 
switch processor will choose an available multicast 
connection number (MCN) and send signal packets to 
update the translation tables in each involved switch 
interface. When the modified packet reaches the next 
multicast packet switch, the switch processor uses 
the tree number in the packet to index a multicast rout- 
ing table, does some necessary checking, and then 
further partitions the destination subset. This proced- 
ures repeats untill all the destinations are reached. 

The concept of multicast connections is a natural 
extension of that of point-to-point connections. That 
is, a multicast connection is a logical association 
among a set of network interfaces over which all pack- 
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ets following the same route, need not carry complete 
destination addresses and arrive in sequence. Based 
on different requirements, four flavors of multicast 
connections are defined over an arbitrary multicast 

5 packet switching network: vanilla multicast connec- 
tions (VMCs), multicast virtual circuits (MVCs), strong 
multicast connections (SMCs) and strong multicast 
virtual circuits (SMVCs). Roughly speaking, vanilla 
multicast connections and strong multicast connec- 

10 tions only provide network-layer connection-oriented 
services to the users, and packet loss is allowed. 
They depend on the users' transport layer to execute 
error control, if necessary. On the other hand, multi- 
cast virtual circuits and strong multicast virtual circuits 

15 provide network-layer virtual circuit services to the 
users, which ensure reliable packet transfer. Theref- 
ore, error control in the transport layer is not neces- 
sary. 

Four flavors of multicast connections are defined 
20 on acyclic subgraphs of the graph representing a mul- 
ticast packet switching networks. Acyclic subgraphs 
guarantee that each multicast connection contains no 
loop and every packet will reach its destination(s) in 
finite time and low delay. 
25 [Definition 3.1]: 

a. An arbitrary multicast packet switching network 
is represented by a graph G = {S,E,L>. where S is 
the set of all multicast packet switches, E is the 
set of all network interfaces, and L is the set of all 

30 links. ^ 

b. G = {S, E, L} represents an acyclic subgraph of 
G, which interconnects all network interfaces in a 
subset E of E via a subsets of S and a subset L 
of L Any link I in L cuts_G intojwo disjoint sub- 

35 graphs G,, u and G,, d . Let E,, u and E,. d be two disjoint 

subsets of E, which contain those network inter- 
faces in G, tU and G, td , respectively. 

c. Each network interface contains a sender com- 
ponent (SC) and a receiver component (RC) that 

40 sends and receives packets, respectively. Let SC 

i and RC i represent the sender component and 
the receiver component of network interface i, re- 
spectively. __ 
Consider an an arbitrary acyclic subgraph G of G. 
45 According to Lemma I, with an LCN chosen for each 
switch interface, any arbitrary data packet flow pattern 
within each multicast packet switch in S can be 
achieved. Interconnection of these individual data 
packet flow patterns viathe links L constitutes a data 
so packet flow partem on G. The flow direction on each 
link is determined by two individual data packet flow 
patterns at its ends. With a data packet flow pattern 
within each multicast packet switches being exem- 
plified, link 3 has a bidirectional flow in FIG. 8(a) and 
55 a unidirectional flow in FIG. 8(c). The corresponding 
transmission matrices are given in FIG. 8(b) and FIG. 
8(d). 

[Lemma 2]:GK/en a G. any data packet flow pat- 

5 
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tern constructed on G has the following properties: 

a. Only a single LCN is associated with each link 
inL 

b. The data packet flow pattern satisfies the weak 
sequencing (WS) condition, that is, point-to-point 
sequential packet transferfrom any network inter- 
face in E to each of its receiving network inter- 
face(s) is guaranteed. 

<Proof>:(a) is clear since, during the construction 
of a data packetflow pattern on G, a common LCN can 
be chosen for the two switch interfaces at ends of 
each link in L (b) holds since each multicast packet 
switch has at least the weak sequencing property. 

[Definition 3^2]: 

a. Given a E, the sending/receiving relationship 
among ail network interfaces in E is represented 
by a N-by-N transmission matrix: TM(E), where N 
is the number of network interfaces in E. TM(E)p,j] 
is 1 if RC j receives packets from SC i, and 0 
otherwise. _ 

b. Given two subsets X and Y of E, the submatrix 
TM(X,Y) is obtained from TM(E) by retaining only 
those sender components in X and^ only those 
receiverjiomponents in Y. Let TM(E| fU , E I(d ) and 
TM(El,d. 5.u) be represented by TM ltU<d and TM W<U , 
respectively. _ _ 
Given a data packet flow pattern on G, at TM(E) 

can be easily obtained by tracing data packet flows 
from each network interface in E. 

By imposing different requirements on data 
packet flow patterns on G, four flavors of multicast 
connections are defined. 

[Definition 3.3]: Given a G, a data packet flow pat- 
tern on G is a vanilla multicast connection, if it satisfies 
the multicast condition: There exists at least one net- 
work interface in E from which the packet stream _is 
destined to two or more network interfaces in E. 
These network interfaces are referred to asmulticast 
sources (MSs). The representation VMC(G) will be 
used to show the relationship between a vanilla mul- 
ticast connection and its associated G. 

The multicast condition implies that (1) At least 
one multicast packet switch in S will duplicate pack- 
ets; and (2) The TM(E), obtained from any VMC(G), 
has at least one row containing two or more I's. From 
this point on, only TM(E)'s having at least one row 
containing two or more I's are considered. Given a G, 
a TM(E) with the weak sequencing condition may not 
be satisfied by a VMC{G). 

[Theorem 3.1]: Given a G, a TM(E) with the weak 
sequencing condition can be satisfied by a VMC(G). 
if and only if it has the following VMC property: For any 
link I in L, if TM, tU ^ (or TM,,^ contains two or more 
non-zero rows, these row must be identical. In other 
words, every sender component in E l ,„ (orE^) seed- 
ing packets to the receiver components in E^ (orEi^) 
must have identical destination subsets of (or E^u). 
<Proof>: The sufficient condition is shown by con- 
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tradiction. Assume that there exists a link I in L so that 
Tty^d contains different non-zero rows. This implies 
that there exist sender components e, and e 2 in E|. u 
and receiver components e 3 and e* in E^ such that 

5 TNKto.eaMe^e*}) is either FIG. 9(a) or (b). In FIG. 
9(a), SC ei sends packets to both receiver compo- 
nent e 3 and e«. and SC e 2 only to RC e 3 . In FIG. 9(b), 
SC e t only sends packets to RC e 3> and SC e 2 only to 
RC e*. Since G is an acyclic graph, there exists a MPS 

10 s in G^ so that packet flows from ^SCs e t and e 2 will 
enter its switch interface A via link I, as shown in FIG. 
9(c), and packet flows destined to RCs e 3 and e 4 will 
leave from switch interfaces B and C, respectively. 
With a single LCN associated with link 1, packets 

15 from SCs and e 2 will have the same LCN in their 
headers when they are sent over linkl. Since one LCN 
only indexes one entry in the packet translation table 
of switch interface A, packets with the same LCN can- 
not be duplicated and routed to different subsets of 

20 outgoing switch interfaces. Therefore, the desired 
data packet flow pattern within MPS s to support the 
submatrices in FIG. 9(a)-(b), can not be achieved. 
This implies that the TM(E) can not be implemented 
by any VMC(G). The above conclusion is also true 

25 when TM|, d , u contains different non-zero rows. 

Next the necessary condition is proved. Let E Uf1 
and E^ (or E^,, and be two subsets of E^ (or 
E ltd ), so that TM(E 41 , E^Xor TM(E lU , E^) contains 
all the 1's in TM, tdiU (or TM ltd , u ). The corresponding 

30 packet flow models of TM(E d . 1 ,Ey r1 ) and TM(E^ 2 .Ed^) 
are shown in FIG. 10, in which MPSs s t and ^ both 
have pattem-1 data packet flow patterns. Let LCN n 
be chosen for link I, then -packets from each sender 
component in E^ and E<, f1 will use LCN n when they 

35 are sent over link I. To achieve these two data packet 
flow patterns, let the routing field in the switch header 
entry of the packet translation table at switch interface 
A (or SI B, resp.), indexed by LCN n, identify a set of 
outgoing links from which packets are transmitted to 

40 the receiver component in E Uf1 (or Ed (2 ). 

Three natural vanilla multicase connections over 
the multicast packet switching network are given 
below. 

a. Point-to-multipoint (Pattem-1): There is only 
45 one multicast source and each multicast packet 

switch in the vanilla multicast connection has pat- 
tem-1 data packet flow pattern. 

b. Point-to-multipoint with upstream capability 
(Pattem-2): There is only one multicaste source 

so and each multicast packet switch in the vanilla 

multicast connection has pattem-2 data packet 
flow pattern. 

c. Multipoint-to-multipoint (Pattem-3): In this vann 
(la multicast connection, each network interface is 

55 a multicast source and each multicast packet 

switch has pattem-3 data packet flow pattern. 
Most data applications require reliable communi- 
cation. To provide a network-based edge-to-edge reli- 

6 
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able service to those multicast applications that 
require completely error-free transmission and that do 
not employ some higher-layer error control protocol, 
the multicast virtual circuit is introduced. 

[Definition 3.4]: A multicast virtual circuit is a vani- s 
lla multicast connection which also satisfies the reli- 
able condition: Point-to-point reliable packet transfer 
from any network interface to each of its receiving net- 
work interfaces is guaranteed. 

There are two issues associated with a multicast 10 
virtual circuit. 

1. Since a vanilla multicast connection may have 
multiple senders, a multipoint-to-multipoint error 
control protocol must be exercised among ail net- 
work interfaces._ 15 

2. Given a TM(E) with the vanilla multicast con- 
nection properties, a VMC(G) can be set up to 
meet the desired information flow relationship 
among users. However, this VMC(G) is only con- 
cerned with transmission of data (or information) 20 
packets, and there may not exist paths in it for 
returning acknowledgements (ACKs). 

One approach to address the second issue is 
described below. If the VMC(G) of a given TM(E) also 
provides paths for returning acknowledgements, it will 25 
be used to transmit both data and acknowledge- 
ments. Otherwise, a_TM'(E) is obtained, where 
TM'(E)p,j] is 1 if TM(E)[i,j] or TM(E)[j.i] is 1. If the 
TM'(E) still has the vanilla multicast connection 
properties, a new VMC(G) is then setup to support the 30 
desired information flow relationship represented by 
the TM(E) and provide necessary acknowledgement 
paths. In both cases, some network interfaces may 
receive undesired data packets and/or acknowledge- 
ments. Therefore, two address fields-the addresses 35 
of the sending network interface and the receiving 
network interface-are reserved in the error control 
header so that each network interface can discard 
undesired incoming packets. The second field is used 
only by the acknowledgements. 40 

A vanilla multicast connection and a multicast vir- 
tual circuit are not concerned with the sequencing 
relationship across multiple senders. Although most 
multicast applications can be supported by a vanilla 
multicast connection or a multicast virtual circuit, 45 
some multicast applications may request the multi- 
cast packet switching network to provide a multicast 
service which maintains the sequencing relationship 
across multiple senders. A strong multicast connec- 
tion and a strong multicast virtual circuit are intn> so 
duced to provide a network-based strong sequencing 
mechanism to those multicast applications that 
require strong sequential transmission and that do not 
employ some higher-layer strong sequencing pro- 
tocol. 55 

[Definition 3.5]: A vanilla multicast connection is 
a strong multicast connection, if it also satisfies the 
strong sequencing (SS) condition: the sequence of 



packet streams arriving at a set of network interfaces 
with two or more common multicast sources are iden- 
tical with respect to the input streams from common 
multicast sources. 

To investigate the relationship between transmis- 
sion matrices with the strong sequencing condition 
and strong multicast connections, consider those 
transmission matrices which have the vanilla multi- 
cast connection properties stated in Theorem 3.1 and 
can be implemented by a vanilla multicast connection. 

[Definition 3.6]: 

a. Given a matrix, its two columns (or two rows) 
a and p are said to be coupled, if column (or row) 
a contains at least two 1 's at the same row (or col- 
umn) position as column (or row) 0. 

b. A matrix with at least two coupled columns is 
said to have the strong coupling property, if its col- 
umns (or rows) can not be partitioned into two dis- 
joint subsets such that there exists one column 
(or row) in each subset and these two columns (or 
rows) are coupled. 

c. A TM(E) having the vanilla multicast connec- 
tion property and at least two coupled columns is 
called a coupled transmission matrix represented 
as CTM(E). 

It is clear that for each non-CTM(E) with the vani- 
lla multicast connection property, its associated 
VMC(G) is_a SMC(G). From this point on, consider 
only CTM(E)s. The problem is summarized below. 

[Problem 3.1]: Given a CTM(E), find the sufficient 
and necessary condition such that its associated 
VMC(G) is also a SMC(G). 

Based on Definition 3.5, the strategy to solve 
Problem 3.1 is to find out all those subsets of receiver 
components which have respective two or more com- 
mon sender components and then check the strong 
sequencing condition for each subset separately. By 
using the following partition method, a CTM(E) can be 
partitioned into a number of submatrices which have 
either no coupling property or the strong coupling 
property. Each submatrix corresponds to a partial 
packet flow of the associated VMC(G). 

[CTM(E) Partition Method]: Let C_be initialized to 
include all the columns of the CTM(E). The disjoint 
subsets {Cj j=0, ... m(>=1)} of C can be generated 
from the following steps as C shrinks to an empty set 

Assume that the subsets C,, ... C H are already 
non-empty and the subset q is empty. 

Step 1: 

If C contains only one column, then move the 
remaining column from C to C 0 . In any case, stop 
if C is empty. Otherwise go to Step 2. 
Step 2: 

Find a column with the largest number of Vs. say 
c, from C. Move c from C to Cj. 
Step 3: 

For each column c in C, repeat the following pro- 
cedure: 
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{If there exists a column in Cj so that these two 
columns have the coupling property, then move c 
from C to Cj. } 
Step 4: 

If q contains only c, then move c from C, to C Q . s 
Go back to Step 1. 

Let E rJ be the set of receiver components corre- 
sponding to those columns in C,. Clearly, 
are disjoint 

By using similar steps as those above, the rows 10 
of each submatrix TM(E, E rJ ), 1^j==m, can be par- 
titioned into a_number of disjoint subsets {R, i = 0. ... 
n (^1)}. Let E»4 represent the set of sender compo- 
nents corresponding to the rows in R} and TM U repre- 
sent the submatrix TM(E,j E rJ ). Clearly, E s>0 , .., B SJ[X 15 
are disjoint 

C 0 and Ro may be empty; however, and R A 
always exist by the definition of CTM(E). Based on 
Definition 3.5, it is dear that the strong sequencing 
condition of the CTM(E) is satisfied , if and only if the 20 
strong sequencing condition of each submatrix 
obtained in the partition algorithm is satisfied. Since 
TM(E,E r ,o) and TM 0j , 1^}^m, rf they exist do not have 
the coupling property, the packetflows corresponding 
to these submatrices are ignored while the strong 25 
sequencing condition of the CTM(E) is checked. How- 
ever, each TM U , 1^i=in, 1=§jsim, has the strong cou- 
pling property. Therefore, to deal with Problem 3.1, 
consider these TM u ,1^i^n, l^jtfm, and deal with 
their strong sequencing conditions separately. Note 30 
that each TM,j has the vanilla multicast connection 
property since the CTM(E) does. Depending on the 
type of multicast packet switches, the sufficient and 
necessary condition that the strong sequencing con- 
dition of a TM|j can be satisfied will be different The 35 
following definition is needed for describing the main 
results which are summarized in Theorem 3.2 and 
Theorem 3.3. _ 

[Definition 3^: Any link I in L_cuts G into two dis- 
joint subgraphs G, fU and G,^. Let E^^u and £, t i Jrd con- 40 
tain those sender components^ in G ltU and G,^, 
respectively. Also let E^jj, u and _Eow contain *ose 
receiver components in G ltU and G^, respectively. 

[Theorem 3.2]: Assume all the multicast packet 
switches are w-MPSs. For any given TM U , 1==i^n. 45 
l£j^m, its strong sequencing condition can be satis- 
fied, if and only if, there exists an inter-MPS link in_L 
such that rf all the sender components in E Sfi are in G^ 
or (G l: d), thenall the receiver components in E rJ must 
be in G l/S (or G, (U ). so 

<Proof>: Since w-MPSs only have the weak 
sequencing property, it is clear that the strong 
sequencing condition among the receiver compo- 
nents in Eij can be satisfied "rf and only if these 
receiver components receive packets via a single link, 55 
where ail packet streams originating from the sender 
components in E^, are multiplexed into a single inter- 
leaved packet stream. In this case, the value of each 



element in TM (J is 1 , since TM (J has the vanilla multi- 
cast connection property and the strong coupling 
property. 

[Theorem 3.3]: Assume all the multicast packet 
switches are s-MPSs. For any given TM u ,1^i£n, 
1^j^m, its strong sequencing condition can be satis- 
fied, if and only if, for any inter-MPS link I in L, there 
do not exist a sender component and a receiver com- 
ponent in G u and a sender component and a receiver 
component in Gw such that these two receiver com- 
ponents have these two sender components as com- 
mon multicast sources. In other words, there dp not 
exist four respective non-empty subsets,_say Esjju, 
E s j4.«j ! _E r , Ji!iU and E rJJfdl of the sets E,,,^, E,,,,,^, E rJJ<u 
and E rJ( i, d , A such that the value of each element in 

TM({E 3 , UfU ,E St g. d }, {Erj.u.E^d}) is 1 . 

<Proof>: When the sender components in E,.j and 
the receiver components in E fJ are connected to the 
same multicast packet switch, the claim is true since 
the s-MPS has the strong sequencing property. 
Therefore, it sufficesto prove the case when the sen- 
der components in E^ and the receiver components 
in E fJ are connected to the same multicast packet 
switches. Since TM g has the vanilla multicast connec- 
tion properties, FIG. 11(a) depicts an equivalent two- 
MPS mode! with respect to each inter-MPS link I. 
Multicast packet switches 1 and 2 represent the two 
multicast packet switches which link I connects. The 
sender components in E,^ and the receiver compo- 
nents in Efj.j.u are connected to multicast packet 
switch 1 , and the sendercomponents in E >iU(d and the 
receiver components in E rJJtd are connected to multi- 
cast packet switch 2. 

The packet flow mode! corresponding to TM(E Stl , 
E rJJ<u ) is shown in FIG.11(b). This is equivalent to the 
case that the sender components in E^ and the 
receiver components in B r j jAJ are all connected to mul- 
ticast packet switch 1, and therefore, the strong 
sequencing condition among the receiver compo- 
nents in E r j JtU with respect to the common sender 
components in E,j is satisfied. Similarly, by removing 
Erjj,u. E»^,d and 1^ from FIG. 11(a), respectively, 
the strong sequencing condition among the receiver 
components in E rJJ ^ with respect to the common sen- 
der components in E^, the strong sequencing condi- 
tion among the receiver components in E^with 
respect to the common sender components in E S| y fU , 
and the strong sequencing condition among the 
receiver components in with respect to the com- 
mon sender components in E^^ are also satisfied. 
Therefore, the only remaining case to check is when 
there exists at least one element in each of four sets 
— E*,u,u> E^u,d, and E^g #u , such that these 
receiver components have these sender components 
as common multicast sources. 

The necessary condition is proved first For any 
inter-MPS link I in L, suppose there do not exista sen- 
der component and a receiver component in G,^ and 
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a sender component and receiver component in G jfd 
such that these two receiver components have these 
two sender components as common multicast sour- 
ces. This implies that the above remaining case does 
not exist Therefore, the strong sequencing condition 
of the TM y is satisfied. Next, the sufficient condition 
is proved by contradiction. Consider FIG. 11(c), in 
which sender component A, sender component B, 
receiver component C and receiver component D are 
in E,,jj, u , E.jj.d. E rJJ , u and E rAW , respectively, and sen- 
der components A and B are common multicast sour- 
ces to receiver components C and D. If each of sender 
components A and B sends a packet receiver compo- 
nents C and D at the same time, then receiver com- 
ponent C (or receiver component D) will receive the 
packet from sender component A (or sender compo- 
nent B) first Therefore, receiver components C and D 
will not receive identical interleaved packet streams. 

[Definition 3.8]: A strong multicast virtual circuit is 
strong multicast connection which also satisfies the 
reliable condition: a set of network interfaces with two 
or more common multicast sources will receive iden- 
tical interleaved packet streams with respect to the 
input streams from common multicast sources. 

By executing multidestination error control 
among all network interfaces in a strong multicast 
connection, some retransmitted packets may exist. 
Without other means, each receiver component may 
put an original packet and a retransmitted packet 
coming from different multicast sources into different 
order, and the strong sequencing condition is viol- 
ated. To solve this problem, each packet in a strong 
multicast virtual circuit carries a global time stamp. 
Time-stamp ordering assures that the strong 
sequencing condition among all the receiver compo- 
nents in a strong multicast virtual circuit is satisfied. 

Various flavors of multicast connections provide 
various kinds of multicast services to users. A multi- 
cast application (MA) is an end-user's use for a mul- 
ticast service or a collection of multicast services. To 
achieve service integration, a multicast application is 
viewed by the users as an integrated service. 

The single-media multicast applications, such as 
multi-destination file transfer, voice conferencing, etc, 
are generally supported by a suitable flavor of multi- 
cast connection. However, for those multimedia mul- 
ticast applications (MMAs) involving two or more 
media, they may or may not be supported by a single 
multicast connection. Consider a multi-party interac- 
tive debugger. Two different scenarios are shown in 
FIG. 12 and FIG. 13, respectively. Each multimedia 
workstation (MMWS) contains a terminal (e.g. T1) and 
a voice bridge (VB) and is connected to a separate 
network interface. In FIG. 12, the editor program used 
for file debugging is a network server in an intelligent 
server node (ISN). Multiple multicast connections are 
used. The dashed lines interconnecting the editor pro- 
gram and the three terminals represent a pattern-2 



multicast virtual circuit with the intelligent server node 
as the multicast source; the dotted lines interconnect- 
ing the three voice bridges represent a pattem-3 vani- 
lla multicast connection. In FIG.13, the editor program 

5 resides in multimedia workstation 1. As before, the 
users can request multiple multicast connections to 
support the multimedia multicast applications; or an 
alternative, the users may request a single pattem-3 
vanilla multicast connection for both data and voice. 

10 The latter results in a multimedia multicast connec- 
tion, in which two media share the same logical chan- 
nel number and are distinguished by different channel 
numbers. Its advantage is that the connection setup 
is done only once. Since the data are also sent via the 

15 pattem-3 vanilla multicast connection, multimedia 
workstation 3 (or multimedia workstation 2) needs to 
discard those incoming editor-program-destined 
editor commands generated at multimedia works- 
tation 2 (or multimedia workstation 3). This data- 

20 packet-discarding is efficiently implemented in 
hardware. Multidestination error control for reliable 
data is executed at the multimedia workstations rather 
than the network interfaces. The voice sharing the 
same multicast connection with the other media 

25 shows an advantage of using distributed voice 
bridges. 

In summary, the number of multicast connections 
needed to support a given multimedia multicast appli- 
cation generally depends on the location of servers 

30 and their relationships. For the local servers, such as 
the editor program and the voice bridges in FIG. 13, 
residing in the multimedia workstations, a single mul- 
ticast connection or multiple multicast connections 
can be requested to support the communication 

35 among them. For the network servers, residing in the 
network interfaces or the intelligent server nodes, 
their locations are generally transparent to the users. 
Therefore, the users may need to request a separate 
multicast connection for each independent network- 

40 based service. 

Multimedia Conferencing 

The description of multimedia conferencing 
45 which follows is based on one particular exemplary 
embodiment, referred to herein as conference system 
1 000, shown in the diagram of FIG. 1 3. Computer sup- 
port for group work can primarily address either 
asynchronization interaction or simultaneous (real- 
so time) interaction among users. Electronic mail is an 
example which primarily addresses asynchronization 
interaction among users. Such systems are most use- 
ful when each user works according to their own 
schedule. For some situations, like crisis handling, 
55 real-time interaction is essential and more productive. 
In a real-time computer based conference, people can 
share information simultaneously through their per- 
sonal computers. The recent advancements in high 
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speed networks and high performance workstations 
make it easier for users to have real-time multimedia 
communications. The medium can be voice, interac- 
tive data, image and video. 

There are several existing real-time conference 
systems. Some of them, e.g. as disclosed in L Agui- 
lar, et al, "Architecture for a Multimedia Telecon- 
ferencing System," Proceedings ACM SIGCOMM'86 
Symposium, August, 1986, H. C. Forsdick, "Explor- 
ations in Real-Time Multimedia Conferencing," Pro- 
ceedings of the 2nd International Symposium on 
Computer Message Systems, IFIP, September, 1985, 
A. Poggio, et al, "CCWS A Computer- Based, Mul- 
timedia Information System," Computer 18, 10, 
October 1985, are constructed as specialized prog- 
rams distributed among the conferees' computers. 
Generally these specialized programs can not invoke 
other applications. Consequently, pre-existing appli- 
cation programs had not to be modified for each con- 
ference system. 

Conference system 1000 (FIG. 13) and a few 
other systems, e.g. as disclosed in *CA. Lantz, "An Ex- 
periment in Integrated Multimedia Conferencing", 
Proceedings of the Conference on Computer-Suppor- 
ted Cooperative Work'86, December, 1986, S.R. 
Ahuja, J.R. Ensor, and D.N. Horn, "The Rapport Mul- 
timedia Conferencing System," Proceedings of Con- 
ference on Office Information Systems, March, 1988, 
allow any pre-existing applications, written for a sin- 
gle-user environment, to be invoked from the within 
frame-work of a conference. These pre-existing appli- 
cations, which produce useful data to a collaboration, 
can be invoked without modifications. These systems 
also allow programmers to develop typical appli- 
cations without having to deal explicitly with the con- 
ference issues. 

Another characteristic of conference system 
1000 is that it is based upon fast packet network 
technology. The transport layer and network layer 
software in system 1000 can support multimedia 
packets (voice, data and image) going through a 
single packet network and can preserve temporal 
synchronization among different media. Temporal 
synchronization is important for real-time multimedia 
communication. For example, a conferee may high- 
light different parts of a text file sequentially while 
making voice comments on highlighted text The high- 
lighted text and the voice comments should appear 
synchronously at all conference workstations. System 
1000 can also record voice and data into a multimedia 
file with temporal synchronization preserved. The 
recorded file can be reviewed by a user asynchron- 
ously or played back in a real-time conference. Voice 
annotation on a recorded file is also supported in sys- 
tem 1000. A user may record a first call while com- 
municating on a second call. 

In addition to invoking pre-existing applications 
and recording a conference, system 1000 also sup- 
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ports changing conference configurations dynami- 
cally. A conference has two basic elements: con- 
ferees and shared applications. A conference 
configuration is changed by changing these two ele- 
5 ments. In system 1 000, a workstation may have mul- 
tiple conferences at one time. A conference 
configuration can be changed by adding or deleting 
applications to a conference dynamically. It can also 
be changed by merging two conferences into one con- 

10 fere nee or splitting a single conference into two con- 
ferences. Thus, the conferees in the resulting 
conference are changed. 

Usually when an application is invoked in a con- 
ference, it should be shared by all conferees in that 

is conference. But in some situations, it may be desir- 
able, to restrict particular conferees from accessing 
the shared application data (either inputting com- 
mands or seeing outputs). The ability to change the 
conference configuration and the ability to control 

20 access to applications greatiy increase the flexibility 
of real-time desktop conferences. 

In the description which follows, a graphical user 
interface (Ul) is presented which allows users to per- 
form these conference functions: setting up a confer- 

25 ence, adding (or deleting) applications, controlling the 
access to an application, recording a conference, and 
merging (or splitting) conferences. 

The graphical user interface is based on an 
important idea— the conference room notion. A confer- 

30 ence room is where the conferees meet and see the 
shared application. If an application is moved into the 
conference room, it will be seen and be used by the 
conferees inside the room. If an application is moved 
out of the room, it will no longer be shared by the con- 

35 ferees. The configuration of a conference is the con- 
tents of the room; the conferees in the room and the 
applications in the room. For any conference, there 
must be some rules concerning who can speak when, 
who has access to sensitive data, etc. This is referred 

40 to as access control. Conferees and applications in 
one room may be moved to merge with another room 
to form a joint conference. A joint conference may be 
split into two separate conferences. In the following, 
basic mechanisms for setting up a conference and an 

45 application are described first The use of menus and 
buttons in the Ul to perform various conference func- 
tions is then described in detail. 

In conference system 1000, the multimedia vir- 
tual circuit, as disclosed in U.S. Patent 4,905,231, 

so issued to W.F. Leu ng et al. on February 27, 1 990, and 
in W.F. Leung, G.W.R.Luderer, M.J. Morgan, P.R. 
Roberts, and S.C. Tu, "A Set of Operating System 
Mechanisms to Support Multimedia Applications," 
Proceedings of 1988 International Zurich Seminar on 

55 Pig'* 31 Communications, March 1988, provides net- 
work layer and transport layer interfaces to the ses- 
sion layer. One important characteristic of the 
multimedia virtual circuit is that all of the media invol- 

10 
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ved in a call are multiplexed into a single virtual circuit 
The network layer deals with the multimedia virtual 
circuit as a whole. In the transport layer, the virtual cir- 
cuit is divided into multiple channels, where each 
channel transports one medium. The primary objec- 
tive of multiplex'ng multiple channels into a virtual cir- 
cuit is to provide temporal synchronization and 
coordination for a multimedia call. 

In system 1 000. each conference has one virtual 
circuit connection. To create a conference, the ses- 
sion layer software first creates a multicast (or point- 
to-point) virtual circuit to connect to remote conferees. 
The newly created virtual circuit has one signalling 
channel. The session layer uses this signalling chan- 
nel to negotiate with its counterparts to agree on the 
successive channels and applications. In system 
1000, one channel is dedicated to one application. 
Multiple applications (channels) can be created for a 
conference. The phone device is treated as an appli- 
cation in the conference model. Data applications can 
be one of the three types: the traditional character-ba- 
sed application programs, the X-based graphical 
based programs and special conference programs. 

For traditional character-based applications pro- 
grams, such as ksh, a driver referred to as a connector 
(disclosed in W.F. Leung, G.W.R. Luderer, M.J. Mor- 
gan, P.R. Roberts, and S.C. Tu, " A set of Operating 
System Mechanisms to Support Multimedia Appli- 
cations, " Proceedings of 1988 International Zurich 
Seminar on Digital Communications, March 1988), is 
used for interprocess communication. It provides 
flexible interconnectivity that can multicast data from 
a source to several selected destinations and concen- 
trate data from several sources to a single destination. 
FIG. 14 illustrates the use of the connector and the vir- 
tual circuit in sharing a character-based application 
program (bin/ksh) by two parties. The process devp is 
needed for transferring a character stream between 
drivers. It is important to note that only one copy of the 
program is executed per application shared by a con- 
ference. This implementation of sharing character-ba- 
sed applications allows ail parties to input commands 
to the application simultaneoulsy. 

In addition to traditional character-based appli- 
cations, graphical applications based on the X win- 
dowing system (disclosed in R. W. Scheifler, and J. 
Gettys, The X Window System," ACM Transactions 
on Graphics , April, 1986), can also be invoked in our 
conference system without modifications. FIG. 15 
illustrates the sharing of a graphical program xprog by 
two parties. The X window servers and xprog are 
unmodified. The pseudo server performs necessary X 
resource identifier translation. The pseudo servers at 
the stations which do not run the application program 
perform the translation. Pseudo servers also control 
the flow of X events from X servers to the application 
program. Unlike the implementation of character-ba- 
sed applications, this implementation only allows one 



party to input at a time. If party 2 has the control to 
input, then the pseudo server will take the event 
stream only from party 2 and feed it to the application 
program. 

5 Data application programs designed specially for 

collaborative work can also be integrated into system 
1 000 as long as they use the X windowing system and 
the multimedia virtual circuit as the transport vehicle. 
As an example, a cut-paste program in system 1 000 

10 allows conferees to share images from workstation 
displays. When a cut-paste application is created, 
each station runs a copy of the cut-paste program. 

Voice applications (phone) are different from data 
applications. Except for voice recording, voice pack- 

15 ets are transmitted to and from the interface hardware 
directly without going through the user space. 

As stated before, one conference has one virtual 
circuit connection. A conference may have multiple 
conferees and multiple applications. To change a 

20 conference configuration, any conferee can dynami- 
cally add or delete an application. When an appli- 
cation is created for a conference, it is shared by all 
the conferees unless access right are restricted. In the 
present embodiment, two conferences can be ^ 

25 merged into one conference. When two conferences • > 

are merged, two virtual circuits are merged into one A 
virtual circuit A conference can be also be split two 
conferences such that each conference has a sepa- 
rate virtual circuit ^ 

30 •'' jj 

Merging and Splitting Multimedia Calls 

The merge operation entails combining two 
MMVCs (multimedia multicast virtual circuits) into one ' 
35 MMVC (FIG. 1). The set of destination end points of 
the resulting MMVC is the union of the destinations in 
the two original MMVCs; after merging MMVC 3 and 
MMVC 9 the resulting MMVC 12 has destinations A, 
B, and C. 

40 Each of the original MMVCs contains one signal- 

ling channel plus other application channels (voice, 
data, or image). The resulting merged MMVC con- 
tains one signalling channel plus the sum of all appli- 
cation channels. That is, if X and Y are the numbers 

45 of application channels in two MMVCs to be merged, 
then there will be X + Y - 1 application channels in the 
resulting merged MMVC. FIG. 1 shows that works- 
tation B has a voice channel and a new data channel 
that did not exist before the merge, and workstation C 

so has a new data channel. 

In the present embodiment, a conference is not 
allowed to have more than one phone. This issue 
arises when two conferences, each with its own 
phone, are merged. At the session layer, each confer- 

55 ence is supported by one MMVC. When two confer- 
ences are merged, their corresponding MMVCs are 
merged at the transport layer. 

In alternative embodiments, applications may 

11 



BMBDOCtD: <£P__SM312BAZJ_? 



21 



BP 0 453 128 A2 



22 



require multiple voice channels. Therefore, this issue 
is resolved at the session layer. It is the session 
layer's job to dose one of the voice channels after a 
merge of two MMVCs each containing a voice chan- 
nel. Of course, the conferees need not be aware of 
these details. To them only one phone appears in the 
conference room after a merge, as is expected. 

When one MMVC is split into two MMVCs (FIG. 
2) the session layer specifies the destination end 
points and the application channels that will exist in 
the resulting MMVCs. Each of the two resulting 
MMVCs will have a subset of the original application 
channels plus one signalling channel and a subset of 
the original destination end points. It is further 
required that each destination end point be a member 
of at least one of the two resulting MMVCs. Similarly, 
each application channel should appear in at least 
one of the two resulting MMVCs. In other words, des- 
tination end points and channels cannot be dropped 
as part of the split operation. 

The originator of a merge operation is required to 
be a member both MMVCs to be merged. Similarly, 
trie originator of a split operation is required to be a 
member of both of the resulting MMVCs. 

User Interface 

There are three levels of windows presented to 
the conferees of a workstation: UI windows, room win- 
dows and application windows. A UI window has 
menus to (1) setup and tear down multiparty (or two 
party) conference calls, (2) add, and delete appli- 
cations in a conference call, (3) merge and split con- 
ference calls, and (4) record a conference. 

There is a room window for each conference call 
from or to a workstation. Each room window contains 
applications and conferees of a conference. Since a 
workstation may have multiple conferences, a room 
window provides a graphical view of each conference 
and its components. Access control is provided 
through a room window. 

There are application windows which appli- 
cations create. Usually an application window con- 
sists of a top level window which contains other 
subwindows inside its boundary. For each application 
in a conference, an application window will appear in 
each station which participates in this conference. 
The application window is the place where shared 
data from applications are displayed and is where the 
commands to the application are input 

When UI starts to run, it does two things: (1) starts 
a listen process to listen to incoming call requests 
from orher workstations, and (2) displays a main 
menu to accept user commands. The main menu is 
shown in the FIG. 16. The main menu has two rows 
of command buttons and a face icon which identifies 
the conferee's workstation. The buttons in the top row 
represent operations on conference rooms and 
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objects in conference rooms. An object may be an 
application or a conferee. The buttons in the second 
row are used to create objects (applications) in a 
room. The command buttons from left to right in the 

5 first row are: room button, quit button, record button, 
trash button, merge button and split button. The com- 
mand buttons in the second row are: directory button, 
phone button, ksh button, X button, and cut — paste 
button. In this user interface, a command button (or a 

10 window) is selected by positioning the cursor on the 
button (or the window) and pressing the left most 
mouse button. The use of the command buttons to 
construct a conference configuration is now des- 
cribed. 

15 To set up a conference, a conferee first creates 

a room by selecting the room button. A room with only 
the originating conferee icon in the room will appear. 
Each room has a room window and a room identifier 
window (which is on the top of the room window). 

20 Each room has a unique room identifier shown in the 
room identifier window. When the directory button is 
selected, UI will display a directory which contains 
face icons (FIG. 18) representing all the people that 
can be reached from this workstation. To make a call, 

25 the originating conferee puts other conferees into the 
room by first selecting conferees and then selecting 
the room window. The UI will initiate a call to the 
remote conferees by opening a multimedia virtual cir- 
cuit connection with conferee addresses as par- 

30 ameters. If the connection is established successfully, 
the originating UI will add the remote conferee icons 
into the room. Otherwise, the room wfll only have the 
originating conferee icon. If a call fails, the originating 
conferee can try again by using the same procedure. 

35 If a call succeeds, the remote site workstations will 
have a conference room which is the same as the con- 
ference room at the originating site. A conference 
room is said to be initially established when a call con- 
nection is established. The first face icon in the 

40 established room is the originator and the remaining 
icons show the other parties that are also in the con- 
ference. FIG. 17 shows an established room with 
three parties and FIG. 18 shows a directory. An 
established room with no applications has only one 

45 signalling channel in the connection. UI uses this sig- 
nalling channel to communicate session layer mes- 
sages for this room. 

After the call has been established, applications 
can then be added into the room. Unlike conference 

so members, applications can be added or deleted 
dynamically. An application is added to a room by first 
selecting the application command button in the main 
UI menu, and then selecting the room. When an appli- 
cation is added to (deleted from) a room, a channel is 

55 added to (deleted from) the existing connection. By 
convention, the first application added to a confer- 
ence room is a phone. Conferees can use voice com- 
munication for negotiating and for coordination the 
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other applications to be added to the conference 
room. A phone icon will appear in the room when a 
phone is added to a room. The phone is initially 
onhook. It goes offhook when the phone icon in the 
conference room is selected. An access control menu 
with phone options is displayed, allowing conferees to 
operate their phones in different modes. 

Only one phone can be added to a room. How- 
ever, many data applications can be added to a room. 
There are two major classes of data applications that 
can be used in the present embodiment: traditional 
character-based applications and graphic-based X 
applications. These applications are typically written 
for a single user. For adding traditional character-ba- 
sed applications, conferees select the ksh button first 
and then the conference room window. A ksh icon will 
appear in the room and a ksh application window 
which is created by the ksh application process will 
also appear on the display. Traditional character-ba- 
sed application can then be invoked through the ksh 
application window. To link the icon to the application 
window, the icon has a unique name assigned by Ul 
which will also be the title of the application window 
and which will appear in the title bar. 

For X applications, a directory menu is provided 
when the X button is selected. Conferees select an 
application from the menu and select the room to 
create the application in that room. Similar to ksh 
applications, an icon and an application window will 
appear when an X application is added to a room. 

The cut-paste button is used to add a cut-paste 
application to a room. The cut-paste program acts like 
a chalkboard which allow conferee to cut images from 
other windows in the display into the cut-paste appli- 
cation window. It also allow conferees to erase (cut- 
out) images from the cut-paste window. FIG. 19 
illustrates a display for a conference room having a 
phone, a ksh application, and an X application (ez 
editor). 

For applications invoked from the ksh button and 
from the X button, a single copy of the program runs 
a the station where the application is invoked. A point- 
to-multipoint channel with upstream capability is 
needed for these applications. For cut-paste appli- 
cations, each station runs its own copy of the prog- 
ram. A multipoint-to-multipoint channel is needed for 
this implementation. 

Any application can be deleted from a conference 
room by selecting the trash button and then selecting 
the application icon in the room. The application icon 
and the application window will disappear when an 
application is trashed. The channel associated with 
that application wfll also be deleted. The trash button 
can also be used to terminate a conference room by 
selecting the room identifier window. When a room is 
trashed all applications are deleted first, then the vir- 
tual circuit connection is terminated. The quit button 
is used to exit the user interface and terminate ail 



ongoing conferences. 

In most conference situations, when an appli- 
cation is shared, all conferees should have the same 
right to receive outputs from the application and to 
5 input commands to the application. In some situa- 
tions, the input or the output of an application may be 
restricted due to security, application features or 
implementation requirements. There are many situa- 
tions that access control will be useful. For example, 
10 one -may share some private data with a subset of 
conferees during a period of time. Another example is 
if one is participating in two conferences, he/she may 
alternatively listen and talk on one conference and lis- 
ten on another conference. The control of a conferee 
15 transmitting to or receiving from an application is cal- 
led access control. The capability for a conferee to 
access a shared application is called the conferee's 
access capability. An access control policy describes 
how to assign access capabilities and how to change 
20 access capabilities. 

There can be many policies for access control. In 
the present embodiment, the transport layer software 
can be requested by the software above to discard 
incoming packets from a particular conferee on a par- 
25 ticular channel. This mechanism is used to implement 
access control. The transport layer is flexible to allow 
other access control policies. 

The access capabaity basically has four levels: 
input and output, output only, input only and no io. For 
30 the phone application, these four levels translate to 
hear and talk, hear only, talk only (can't hear) and 
neither hear nor talk. We define a strict order on the 
degree of control on these four levels: input and out- 
put has the highest order and no io has the lowest 
35 order. Our access control policy is as follows. When 
an application is added to a conference room, each 
conferee is given a initial access capability and a ceil- 
ing access capability based on the type of the appli- 
cation. Fora phone application, the initial capability is 
40 no io for each conferee (equivalent to onhook) and the 
ceiling capability is input and output For ksh and cut- 
paste applications, each conferee has input and out- 
put initial capability and ceiling capability. For X 
applications, the controller has input and output initial 
45 and ceiling capabilities. All others have output only ini- 
tial and ceiling capability. The capability levels 
implemented for each applications are different from 
application to application. Phone has all four levels. 
Ksh and cut-paste has three levels: input and output, 
so output only and no io. X applications only have two 
levels: input and output and no io. 

Only the controller can change another con- 
feree's ceiling capability. When the controller moves 
a conferee's current capability, the new level 
55 becomes the ceiling capability of that conferee. 

A conferee can change his/her own current 
capability to a lower or a higher capability level. A con- 
feree can move to a higher level as long as that level 
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does not exceed his/her own ceiling capability. When 
a phone application is added to a room, it is initially 
onhook (no io). Each conferee needs to change the 
capability to input and output in order to have voice 
conference. 

To change the capability, a conferee first select 
the application icon in the conference room. An 
access control window will appear. The access con- 
trol window shows ail the conferee icons and all poss- 
ible access capability levels are shown on the right 
side of the conferee icon. The current access capabi- 
lity level (not the ceiling capability) is highlighted. FIG. 
20 and FIG. 21 show the access control windows of 
phone and ksh respectively. By selecting the access 
capability level provided in the access control window, 
a conferee can change the current access capability. 
The permission to change capabilities is always 
checked by the UI according to the policy mentioned 
above. If the change is not permitted, the current 
capability will not be changed. Conferees can always 
select any application icon to review the capabilities 
of that application. For the phone application, the cur- 
rent capability is also reflected in the phone icon in the 
conference room. 

The record button provides real time conference 
recording, multimedia document review and anno- 
tation. When the record button is selected, a play-re- 
cord menu window will be displayed (FIG. 22). The 
play-record menu window has five options: play, 
record review, anot and stop. To record a conference, 
it is assumed that the conference room has a phone 
application and a ksh application. When a conferee 
selects the record option, a query window will appear 
to input the file name to be recorded to and the ksh 
application to be recorded from. To start the recording 
process, the conferee needs to give the file path name 
(or use the default file name) and selects the ksh 
application icon from a conference room. The voice 
packets from the phone and the data packets from the 
ksh application are multiplexed together and stored 
into the given file. The temporal relationship is main- 
tained when this multimedia file is played back. 

The stop button is used to stop a conference 
recording. When stop is selected, the play-record 
menu window will disappear. To play back a recorded 
file, the record button is used to display a play-record 
window and then the play option is selected. Similar 
to the recording, a query window is popped up to input 
the file name and the ksh to play the multimedia file. 
When playing a file, text data will appear on the ksh 
application window and voice packets will be sent to 
the voice handset The play option is used to play a 
recorded file into a conference room, so it assumes 
that the room has a voice application and a ksh appli- 
cation. A review option is provided to let a conferee 
review a document locally. When review is selected, 
a query window will be displayed for a conferee to 
input the file name and a ksh window will be created 



to review the given file. The ksh application does not 
belong to any conference room. During the review 
process, voice annotation can be invoked by selecting 
the anot option. Selecting the anot button will start the 

5 annotation. Voice packets will be inserted into the 
multimedia file. Selecting the anot again will stop the 
annotation. Voice annotation can also be made while 
playing a recorded file. 

One major design consideration of the user inter- 

10 face concerns performance of real time conference 
operations. In addition to adding/deleting applications 
to/from a conference, the conference operations pro- 
vided are: merging two conferences into one confer- 
ence, and spirting one conference into two 

15 conferences. The notion of a conference room is used 
to represent a conference. Each conferee and each 
application involved in a conference are represented 
in the conference room by a icon. For desktop confer- 
ences, the conference room representation is a con- 

20 venient way to perform real time conference 
operations. 

When a conferee want to split a conference, 
he/she first selects the split button and then selects 
the conference room to be split by selecting the room 

25 identifier window. UI will display a splitting menu win- 
dow and a new conference room. FIG. 23a illustrates 
a splitting menu, FIG. 23b a room which is being split, 
and FIG. 23c a new conference room. The new con- 
ference room will have a new identifier and the origi- 

30 nating conferee icon in it A splitting menu window has 
four selections: move, copy, abort, and done. 

An application or a conferee can be copied or 
moved into the new conference room. When a con- 
feree is copied, it means the conferee will participate 

35 in both the original conference and the newly created 
conference. When an application is copied, the origi- 
nal application process will be kept for the original 
room and a new application instance will be created 
for the new conference room. The contents of the 

40 original application window will not be copied into the 
new application window. When a conferee is moved 
to the new room, he/she will no longer remain in the 
original conference. When an application is moved 
into the new room, the original application may be 

45 deleted from a conferee's workstation if the conferee 
is not in the new room. The originating conferee uses 
move and copy selections repeatedly to construct the 
new configuration of the original conference room and 
the new conference room. The abort menu aborts the 

so split process. The done menu informs UI to start the 
splitting process. The UI will not call the network to 
split the virtual circuit connection unless the done 
menu is selected. The originating UI needs to inform 
each remote UI how the conference room will be split 

55 before or after splitting the virtual circuit Voice com- 
munication is assumed to be important for conferees 
to coordinate both the split and the merge operations. 
If the network split operation is successful, the 

14 
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original connection will be split into two connections. 
In the connection for the original room, the channels 
used by the applications which have been moved are 
closed, in the connection for the new room, new chan- 
nels are created if new application instances are 
created. FIGS. 24 AND FIG. 24b show the room con- 
figurations after the phone is copied, and the ksh 
application and a conferee are moved from the origi- 
nal room. 

Merging is an inverse process of splitting. To 
merge two conferences, the originating conferee first 
selects the merge button and then selects the two 
rooms to be merged in order the merge-from room 
and the merge-to room. When two conference rooms 
are merged, all conferees in these two rooms will be 
merged into the merge-to room. Except for the phone, 
ail applications from the merge-from room will be 
added to the merge-to room. If both rooms have 
phones, then only the phone in the merge-to room is 
kept If a station only has the merge-from room, the 
merge-from room will be modified into the merge-to 
room. This means that applications already running 
will be kept (except possibly phone application) and 
applications from the merge-to room will be added. If 
a station only has a merge-to room, the Ul will perform 
a similar task. If a station has two rooms, ail the appli- 
cations will be kept (except only one phone channel 
is kept). 

Merge and Split Protocol 

In one alternative implementation of a merge and 
split protocol, a merge or split request is transmitted 
into the network, and the request is propagated 
through the network nodes, finally delivering the 
request at all involved workstations. However, the 
merge and split operations cause alterations in the 
connectivity of the original MMVC(s) and the data 
structures associated with the MMVC(s). If a 
merge/split operation is committed to using a single 
phase protocol, as described above, then it is imposs- 
ible to ensure that failure of the operation will not dis- 
rupt the original MMVC. For example, envision a 
merge request sent into the network. One node in the 
networks accepts the request, updates it data struc- 
tures and forwards the request on to the next node. 
Unfortunately, the second node does not have the 
resources to complete the merge and thus denies the 
request. In this case the original MMVCs can not be 
recovered, since the first node has already committed 
to the operation; there is no alternative butto clearthe 
MMVC. This is disastrous for the user, not only is his 
request denied but his previously stable connection is 
destroyed. 

For this reason, the preferred embodiment is a 
two phase protocol. The protocol minimizes the effect 
of a failed operation on the user. The first phase of the 
protocol is used to determine if all components, that 



is work-stations and network nodes, agree to com- 
plete the merge or split operation. When the works- 
tation originating the operation determines that all 
components have agreed to the operation, it initiates 

5 the second phase of the protocol by issuing an accept 
message. The accept message indicates that all com- 
ponents should complete the merge or split operation. 
If any component denies the requested operation dur- 
ing the initial phase, then the original MMVC(s) are 

10 unaffected. 

The preferred protocol is shown in FIG. 25. The 
"mrg/splt— inquiry" is a session layer to session layer 
communication. The "iocti (mrg/split)", represents an 
iocti call made by the session layer requesting a 

15 merge or split be initiated by the transport layer. 

Once the transport layer receives a merge or split 
command via an iocti call, it sends a merge or split 
request (denoted wnMrg/wnSplt in FIG. 25) into the 
network; the latter acknowledges the receipt of the 

20 packet When a wnMrg or WnSplit packet is received 
at an endstation, it is immediately acknowledged by 
the transport layer with a wnAck packet This only indi- 
cates that the packet was received at the endstation. 
The receipt of a wnMrg or wnSplit also causes a ses- 

25 sion layer listen process to be given a merge or split 
request message. The listen process in turn issues a 
merge or split iocti call on the far end workstation. The 
transport layer of each far end workstation then sends 
and end-to-end acknowledgement to the originating 

30 workstation, indicating that they agree to the 
merge/split request 

Once the originating workstation has received an 
end to end acknowledgement from all participating 
workstations, a wnMaccept or wnSaccept packet is 

35 issued. The accept packet indicates that all ends- 
tations have acknowledged receipt of the merge or 
split request and each station agreed to the merge or 
split command. This packet acts as a commit; after 
being sent the merge or split command can not be 

40 aborted. Upon receiving this packet the far end work- 
stations send an end-to-end acknowledgement and 
then return from the iocti call. The second set of end- 
to-end acknowledgements are used to indicate when 
the originating workstation can return from the iocti 

45 call. Once these acknowledgements are received all 
connections are established and packet transfer can 
begin immediately. 

It is to be understood that the above-described 
embodiments are merely illustrative of the principles 

so of the invention and that many variations may be dev- 
ised by those skilled in the art without departing from 
the spirit and scope of the invention. It is therefore 
intended that such variations be included within the 
scope of the claims. 

55 
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Claims 



said second call further comprises a signaling 
channel, and said plurality of channels further 
includes a signaling channel. 

7- A method in accordance with claim 4 wherein at 
least one of said first and second calls further 
comprises an image channel, and said plurality of 
channels further includes an image channel. 

8. A method in accordance with claim 4 wherein at 
least one of said first and second calls further 
comprises a video channel, and said plurality of 
channels further includes a video channel. 



9. A method in accordance with claim 1 wherein said 
plurality of channels comprise virtual channels of 
a multicast connection. 

10. A method in accordance with daim 1 further com- 
prising in response to user input, selectively pro- 
viding said at least three user stations with 
access to transmit information to and receive 
information from said plurality of channels. 

11. A method in accordance with daim 1 wherein at 
least one of said first set of user stations and at 
least one of said second set of user stations 
include display means, said method further com- 
prising 

in response to user input at said one of 
said first set stations, both establishing said first 
call in said network and displaying represen- 
tations corresponding to said first set of user sta- 
tions in a window on said display means of said 
one of said first set stations, and 

in response to user input at said one of 
said second set stations, both establishing said 
second call in said network and displaying repre- 
sentations corresponding to said second set of 
user stations in a window on said display means 
of said one of said second set stations. 

12. A method in accordance with daim 1 wherein one 
of said plurality of user stations is a member of 
both of said first and second sets, said one user 
station comprises display means, said merging is 
in response to user input at said one user station, 
said method further comprising 

further in response to said user input at 
said one user station, displaying representations 
corresponding to user stations of said first and 
second sets in a window on said display means. 

13. A method in accordance with claim 12 further 
comprising 

further in response to said user input at 
said one user station, displaying, in said window, 
representations each corresponding to one of 
said plurality of channels. 

14. A method in accordance with daim 12 wherein 
said plurality of channels indudes at least one 
data channel, said method further comprising 

50 displaying, in a second window of said dis- 

play means, a data communication application 
shared among user stations of said first and sec- 
ond sets via said one data channel. 

55 15. A method in accordance with daim 1 wherein one 
of said plurality of user stations is a member of 
both of said first and second sets, said method 
further comprising 



1. A method for use in an arrangement comprising 
a communication network interconnecting a 
plurality of user stations, said method comprising s 

providing a first call among a first set of 
said user stations, 

providing a second call among a second 
set of said user stations and 

merging said first and second calls into a 10 
single call, comprising a plurality of channels, 
among at least three user stations from said first 
and second sets, said plurality of channels corre- 
sponding to a combination of channels of said first 
and second calls. is 

2. A method in accordance with daim 1 wherein said 
merging comprises 

merging said first and second calls into a 
single call connection, comprising said plurality of 20 
channels, among the union of said first and sec- 
ond sets of user stations. 

3. A method in accordance with daim 1 wherein said 
first call comprises a signaling channel, said sec- 25 
ond call comprises a signaling channel, at least 
one of said first and second calls further com- 
prises at least one non-signaling channel, and 
said plurality of channels comprises a signaling 
channel and at least one non-signaling channel. 30 

4. A method in accordance with daim 1 wherein said 
first call comprises at least one data channel, said 
second call comprises at least one data channel, 
and said plurality of channels includes a data 35 
channel for each data channel of said first and 
second calls. 

5. A method in accordance with claim 4 wherein at 
least one of said first and second calls further 40 
comprises a voice channel, and said plurality of 
channels furtherindudes a voice channel. 

6. A method in accordance with daim 5 wherein said 
first call further comprises a signaling channel, 45 
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in response to user input at said one user 
station, transmitting a merge request via said net- 
work to each of the other user stations of said first 
and second sets, and 

in response to receipt of a favorable reply 5 
to said merge request from each of said other 
user stations of said first and second sets, said 
one user station initiating said merging. 
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